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Abstract —In this paper, we propose a novel formulation to 
understand the tradeoff between binning cost, meta informa¬ 
tion leakage and hypotheses distinguishability in communica¬ 
tion networks. To do so, we reiterate the idea of partitioning 
random items from our previous work. 

Under this framework, the goal is to maximize the measure 
of distinguishability between two hypotheses while restraining 
a certain level of information leakage and overall binning cost 
by properly dividing a set of M random items into N bins. 

To do so, we formulate a novel multi-agent multi-variate 
optimization problem which are generally NP-complicated. 
We then utilize the submodular nature of the problem to find 
sufficient conditions to (1) secure the existence of a solution 
to our problems; (2) lower the complexity of the problem at 
the cost of some accuracy. After offering sufficient conditions 
to secure the existence of a solution in all general cases, we 
offer examples which help signify the importance of using 
submodular properties. 

Index Terms —partition, information leakage, privacy, mu¬ 
tual information, hypotheses distinguishability, submodularity 


I. Introduction 

The concept of a tradeoff between an overall binning 
function and meta information leakage in a communication 
network is one of the most recurring subjects in modern 
engineering research. In this paper, we aim to offer a novel 
insight into and a novel formulation of such problems. 
To motivate such concerns, we choose to first provide an 
example about the necessity of considering such tradeoff. 

Through our use of internet, there are many times when 
we wish to be distinguished from other users (so as to 
enjoy a tailor-made service) while limiting the amount of 
information reflected to an eavesdropper by patterns shown 
in sequence of webpages we have visited. 

One method to carry this out, is the use of Virtual Private 
Network (VPN)s. Such networks offer the user complete 
anonymity by allocating a fake IP address to the user 
rendering him untraceable to the eavesdropper. Thus, if 
we could address all our browses through the same VPN, 
no information would be leaked. However, a VPN tends 
to slow down the connection speed . Also, the pace is 
further dropped when more traffic (other users’ data) is 
imposed upon the network. As a solution, we can employ 
multiple VPNs and thus cut down on the utility loss. 
However, by using multiple VPNs we are allowing a level 
of privacy breach into our browses where the eavesdropper 
is able to deduce some information about our tendencies by 
observing the distribution of used webpages over multiple 


networks. Then, if a utility function based upon connection 
speed -bandwidth- for the user is calculable, a privacy 
constrained problem between the user and an eavesdropper 
(for example a service provider) could be defined. 

The solution to such a problem could offer insights in 
regards to the tradeoff between VPN allocation utility and 
meta information leakage when we face the problems of 
partitioning a set of random items (i.e. websites that a user 
has chosen to visit following his own distributions) into a 
given number of bins (i.e. a given set of VPN servers each 
of which has its own utility function, as will be further 
detailed in Section II). 

In our proposed framework as detailed in Section II, 
meta data information refers to the patterns about a se¬ 
quence of items (for example the user’s favorable websites) 
infer-able based upon a sequence of bins (e.g. proxy 
sites) observed by an eavesdropper. This assumption is an 
expansion of [1] and [2] and our own work in [3], 

Furthermore, in our investigation we find it necessary 
to introduce a measure of distinguishability and a binning 
cost function. 

Finally, we introduce multi-submodularity and submod¬ 
ularity as two means of reducing the complexity level of 
such problems, namely, dividing M random items into N 
bins, under an upper-bound on leaked meta information. 

The concept of privacy has already been explored in 
many works such as [4], [5] where a general but non- 
mathematical explanation was offered. However; in our 
work, we go into further details as to what privacy rep¬ 
resents in our framework and how it could be formulated 
into many settings. Later, we find it necessary to utilize 
the concept of multi-submodular set function problems 
and their solutions. This concept was widely discussed in 
[6] where they introduced a series of sufficient conditions 
on multi-submodular set functions by which the multi¬ 
submodular problem could be transformed into a submod¬ 
ular set function problem. Then, further discussions about 
the existence of a solution to the new problem were made. 
By doing so -and if a solution were proven to exist-, the 
complexity of the problem could be shown to be reduced 
from NP to polynomial. However, [6] did not offer any 
algorithmic solutions in such cases which unfortunately 
results in us simply proving the existence of a solution 
rather offering an algorithm to support such solution as 
well. 

As for the addition of distinguishability, to the best of our 
knowledge, the closest work to ours was done by [7] where 



they considered the tradeoff between distinguishability and 
information leakage when the former is quantified using 
KL-distance and the latter using mutual information. 

It is important to note that some of the concepts revisited 
in this paper, were already well-defined in our previous 
work [3], Thus, in this paper, we refer the reviewer to 
[3] for some problem formulations and instead attempt to 
address the problem concerning distinguishability in full 
details. 

As for the novel problem addressed in this paper, our 
goal is to offer (1) a measure of distinguishability be¬ 
tween the K hypothesis; (2) a measure of average leaked 
information given any of the I\ hypothesis is active; 
(3) a multi-agent multi-variant optimization problem with 
privacy leakage and overall binning cost constraints; (4) 
insight into the complexity of such an TVP-hard problem 
and assumptions we may need to incur to simplify it into 
a polynomial problem such as aiming to minimize the 
upper bound or the form of the cost function; (5) a full 
description of the algorithm utilized to find the solution 
given the sufficient conditions followed by the proximity 
results and finally (6) examples to further demonstrate the 
applicability of submodular solutions in our own and even 
more general problems of such nature. 

The rest of this paper is organized as follows. In Section 
II we formulate the two problems in terms of privacy 
and utility functions. We dissect what the goal and the 
constraints are. In Section III, we inspect the overall utility 
functions of each problem and then find the sufficient 
conditions under which they are equipped with multi- 
submodular property. Due to general limited knowledge 
about multi-submodular solutions and how they are devel¬ 
oped, we then assume a specific case N = 2. In Section 
IV two algorithms for each case are developed which offer 
a polynomial complexity and an accuracy level of e or - 
depending on whether the goal is to minimize or maximize 
the overall utility function respectively. 

II. System Model 

In this section, we aim to formulate the problem of 
distinguishing two possible hypotheses while keeping the 
average leaked information and the overall binning cost 
to two specific thresholds. To do so, we assume access to 
two multinomial distributions G\ and G 2 each with prior 
probabilities P(Gi) and P{G 2 ) = 1 — P{G\) defined 
over M items. These multinomial distributions impose 
probabilities 77 over item i given multinomial distribution 
G p is chosen where p £ {1,2}, i £ {l,--- ,Mj. It also 
follows that Y^L 1 7Tj f = l,p = 1,2. 

We then aim to distinguish these two distributions by 
choosing binning as our method of observation meaning 
we attempt to find an M-to-N mapping between the set 
of M items and set of N outputs which can best help us 
decide which multinomial distribution was chosen. 

We further assume that binning the M items is a costy 
function whose amount of cost is based upon the sum 
of overall probabilistic weight of items mapped to each 
output respectively. We further note that by imposing a 
binning procedure over the set of M items we are leaking 


a certain amount of information about them (seeing as how 
frequencies of their activity is exposed). 


A. Set Allocation 

First, we propose an abstract framework to formalize 
the goal of seeking partition of M items into N bins. 
More specifically, we aim to allocate each one of 1 < 
i < M possible items to one of N output bins. There 
could be at most N M such partitions. It follows that 
any set allocation Ai , 1 < l < N M results in N sets 
Sj l) C {1, 2,..., M}, j = 1,2, Each such set is 

defined as 

Sf = {i|6»S = 1} 

whm (i) 

We further assume D =$,j^k. Furthermore we 
have (J^_ 1 Sj 1 ' 1 = {1,2, Finally the size of each 

such set Sj 1 ' 1 is defined as L'j 1 . 

B. Probabilistic Model 

We assume at any time slot one and only one of 
the inputs is chosen with a certain probability. Thus, 
assuming hypotheses G p is active and if we use variable 
X £ {1,2, as a representation of set of items, we 

could have P(X = i\G p ) = P( = 1) = n ijP ,l < 
i < M,,p £ {1,2} as a representation of the probability 
of choosing item i from the set X under hypotheses G p 
where 7^ £ {0,1}. It further follows that Y^iLi 7 i p = 
1 ,p £ {1,2}, stipulating that one and only one of M items 
is selected. 

Next, we introduce an observable random variable Y £ 
{1,2, ••• ,7V}, denoting the index of the bin (the proxy 
site) employed to carry one of the M > N items. It follows 
that the probability of each bin’s appearance given a set 
allocation scheme such as Ai under hypotheses G p will be 
equal to 

P(Y = j\A l ,G p ) 

M 

= Y J P{Y = j\A u X = i, G p )P(X = i\ A U G P ) 

i— 1 

M 

= Y J P(Y =j\AuX = i)P(X = i\G p ) (2) 

i=1 

Furthermore, P(Y = j\Ai,X = i) = 6^ £ {0, ,1}. It 
thus follows that 

M 

P(Y = j\A u G p ) = £ 0%P(X = i\G p ) -> 

i =1 

P(Y = j\A l ,G p )= Y, = a f P) (3) 



C. Revealed Information 

By choosing to allocate M items to N bins where 
N < M, we have injected ambiguity and uncertainty into 
the output binning index sequence about the input item 
sequence over a successive n visits or channel uses. In 
other words, if we originally chose to transmit n of such 
items, our total set of possible sequences would be of form 
= [. XiX 2 ...X n ] out of M n possible outcomes. From an 
observer’s perspective which can only have access to which 
one of N bins is deployed in each time slot, sequences in 
the form of Y 1 "' = [Yj Y 2 ... Y n ] has cardinality of at most 
N n < M n . Despite the amount of uncertainty added due 
to the many-to-one binning, the output sequence sill reveals 
certain amount of information regarding the patterns of 
sequences of M random items. 

This observation could be further studied by indicating 
how our allocation system resembles a coding framework 
where we have an equivalent channel whose input variable 
is X and output Y, as demonstrated in Figure 1. 



Figure 1. Coding Channel Representation of the Problem 

Under such a framework, the equivalent channel output 
sequence Y can help an eavesdropper classify the input 
sequence X into a number of differential classes. As a 
result, information about the specific input item patterns 
is leaked to certain degree and can be measured using 
conditional mutual information I(X; Y\Ai, G p ) between 
A' and Y, under a hypotheses G p given a particular binning 
(i.e. partition Ai relationship as illustrated in Figure 1. 

Such conditional mutual information thus measures the 
maximum number of bits of meta information about item 
sequence per channel use. Therefore, we can have at most 
2 ni(x-\ I Ai,g p ) se q Uences X" distinguishable by inferring 
based on Y". We thus adopt I{X\ Y\Ai, G p ) as the privacy 
metric conditioned on a particular partition mapping Ai 
under a specific hypotheses G p . It follows that due to the 
combinatorial nature of a set allocation problem there are 
a total of N m possible methods to allocate these M items 
to the N sets. 

We can formulate the mutual information over a set 
allocation Ai, 1 < l < N M under hypotheses G p as: 

I(X ; Y\A U G P ) = H{X\A l ,G p ) - H(X\Y, A h G p ) = 
H(Y\Ai,G p ) — H(Y\X, A h G p ) = H{Y\A U G V ) = 

tt!A bp) A l ’P) Ahp)\ tA\ 

where we have used the notion of H(ai, a 2 ,a m ) = 
— a v log civ and the fact that H(Y\X, A[,G p ) = 0 
because if the input X and the channel scheme Ai and 
hypotheses G p are known, then output Y will offer no 
uncertainty. 


D. Distinguishability Measure 

In this section, we aim to define a measure which 
can evaluate the distinguishability between a number of 
multinomial distributions on M items. 

Given that a certain set allocation Ai has been chosen, 
depending on the choice of G p , two different distributions 
defined by using variables a < j’ p could be developed where 
we have: 

ofhv) =P( Y =j\A l ,G p ),pe{l,2},je{l,...,N} ( 5 ) 

where the two probability sets Pi = {o^’ 1 ^, j = 

1,2and P 2 = {otj’ 2 \j = 1,2,...,TV} represent 
the multinomial distributions under allocation scheme l 
given two prior probability distributions G\ and G 2 which 
we aim to distinguish. To evaluate the distinguishability 
between two such distributions, we utilize the definition of 
symmetric KL divergence: 

t^{KL(Pi\\P 2 ) + KL(P 2 \\Pi)} 

= (“j U) ~ af 2) )(loga^’ 1) - log a ( ] l ’ 2) ) (6) 

1 2=1 

E. Cost Function 

In the problem of hypotheses distinguishability, we note 
that we are utilizing binning as a means of detecting the 
multinomial distribution imposed upon the items. It thus 
makes sense that binning would be a costy process. We 
assume that the cost function is defined as the probabilistic 
sum of respective set functions based upon items allocated 
to each bin. In other words, we assume given a set 
allocation Ai and that G p is selected, the cost function 
follows the following format 

C{A l ,G p ) = Y,<*f P) f(Sf p) ) (7) 

2=1 

where represents the set of items allocated to bin 

j given that hypotheses G p is active. We thus aim to 
minimize the average of C(Ai,G p ). 

CiAt) = P(G P ) a?’ P) /(S?’ P) ) (8) 

P = 1 2 =1 

F. Problem Formulation 

As was mentioned in the beginning of this section, our 
overall goal could be summed up in finding a partition of 
M items into N bins under which we can maximize the 
distinguishability between hypotheses G\ and G2 while 
keeping the overall leaked information and cost function to 
a threshold. We thus formulate the following constrained 
optimization problem: 



N 


mm 

1<KN M 


N 


EE P ( G ^[ a ? /( 5 i ) 


p= i j =i 

£ 
3 


—\ 2 ol'-' p) loga[ l ’ p) ] 


( a f l) - a j i,2) )( lo g a j’ 1] - lo S a j' 2) ) (9) 


3 = 1 


while making certain that 

N 2 

E E P(G P )[-af p) log {af p) )\ < I th (10) 

3=1 P=1 

and 

2 N 

E E P{G P )af p) f( s f p) ) <C (11) 

P= 1 3=1 

hold true. It is important to note that the variables Ai 
and A 2 represent the weight offered to each constraint in 
comparison with the objective function. Thus, the problem 
formulated in Eq.(9) is not a Lagrange multiplier problem 
where we aim to find the optimal values for Ai and A 2 ; 
they are simply given to us by the user. 

III. Multi-Submodular Set Functions as a 
Means of Solution 


Since in our formulation functions are separately defined 
on different sets, the condition in Eq. (12) is simplified to 
the sufficient condition of submodularity of: 

F(sj°) = P(G i)[af l) f{Sf l) ) - \ 2 af l) logaJ U) ] 
+P{G 2 )[af 2) f{Sf 2) ) - \ 2 af 2) logaf 2) ] 

- af 2) )(log af 1] - log af 2] ) = F(Sj) 

( 13 ) 

for all sets ,Sj. Finally, in order to further simplify the 
formulation of the problem we aim to drop l from the 
above definition and rewrite: 

F{Sj) = P(G 1 )[«S 1) /(Sj 1) ) - A 2 af ) log a? 5 ] 
+P(G 2 )[aff(Sf) - A 2 af log af) 

-y (af } - af^logaf 1 - logaf } ) (14) 

The new denotation alludes to the fact that once a set 
allocation Ai is chosen, its index could be dropped. 

In the next step, we opt to use diminishing return prop¬ 
erty as the means of making certain each of these functions 
is submodular. Following is a definition of diminishing 
returns for submodular functions. 

Diminishing Property Return dictates that if we define 
S as the universal set, a set function F : 2 s —> R + is 
submodular if, for all A, B C § with A C B and for each 
x € S — B we have [8]: 


Unfortunately, the problem formulated in Eq.(9) is NP- 
complicated (it is only solved when a search over N M 
possible set allocations is fully carried out and the optimal 
set allocation is revealed). Still, we could opt to utilize 
the definition of multi-submodular set functions so as to 
reduce the complexity to that of polynomial at the cost of 
accuracy. 

In other words, we aim to introduce a model where 
the user inputs prior probability distribution of different 
hypotheses, item distribution under each hypotheses, the 
cost set function, the leaked information threshold I t h , 
the overall binning cost threshold C and the constraint 
weights Ai and A 2 and we decide whether a semi-optimal 
set allocation scheme for such a setting could be found. 

A. Imposing Multi-submodularity 

In [6], it was shown that if we can prove multi- 
submodularity for functions such as those formulated in 
Eq. (9), then they could be modeled as simpler problems 
(submodular set functions). We thus, aim to find the 
sufficient conditions for such occurrence. To do so, we first 
offer a review of multi-submodularity. 

As mentioned in [6], if we define M = {1, 2,..., M}, 
then a multivariate function F : (2“)^ —» M+ is multi- 
submodular if for all pairs of tuples (,S)..... ,S\v) and 
(Ti, ...Tat) <E ( 2 M ) N we will have: 

F(Si,S N ) + F(T U ...,T n ) > F(Si U Ti,..., S N U T N ) 
+F(S 1 nT 1 ,...,S N nT N ) (12) 


F(A U {x}) - F(A) > F(B U {x}) - F(B) (15) 

Note: For any further references, we first need to address 
a series of variable and function definitions which are going 
to play a vital role in the rest of this chapter: 

Definitions for Problem 1 

1. Any variable represented with a capital Letter repre¬ 
sents a set. 

2. Any variable represented with a small letter repre¬ 
sents an element. 

3. A — B represents a set containing all elements of set 
A which do not appear in set B. 

4. a x represents the probability of item x and cha 
represents the sum of probabilities of items mapped 
into a set A. 

5. Ob\a represents the difference in the sum of proba¬ 
bilities of items mapped into the sets B and A which 
could be further shown as Ob\a = cxb — ola- 

6. g(C,D) represents the 1 st order difference of a set 
function f(C) from f(C — D ) where D C C which 
could be formulated as g(C, D ) = f(C) — f(C — D). 

7. q(C, Ci, D, Di) represents the 2 nd order difference 
of a set function /(C) where ft C C and D\ C 
D which could be formulated as q(C, C 1 , D, D 1 ) = 
g{C,D)-g(Ci,D 1 ). 

8. We assume the probability of items is sorted in a 
decreasing manner such as 7T1, > Tr 2l > ... > 7Tmi 
over Hypotheses Gi and 7 Ti 2 > tt 2 2 > ... > 7 tm 2 
over Hypotheses G 2 . 


Next, we find sufficient conditions for multisubmodular¬ 
ity of the overall utility function described in Eq. (9). 

At a first glance, it seems that the function F(Sj) = 
F(Sj 1 ' 1 ) has a singular relationship with set S'p. However; 





there is a secondary relationship the function shares with 
the set Sj l ^ C = S — Sp where S represents the universal 
set. This relationship could be modeled as 

F(sf c ) = P(Gi)[(l - p\ l ' 1] )f(S - Sp 1)C ) 
-A 2 (l-^’ 1) )log(l-/3f 1) )] 
+P(G 2 )[(l-pp 2) )f(S-Sp 2)c ) 
-A 2 (l-/f' 2) )log(l-/f' 2) )] 

~(Pp 2) - Pf’^Qog (1 - Pp t) ) - log (1 - pp 2) )) 

(16) 

where we have used the fact that pP = 1 — ap seeing as 
how we define 

PP = E ^ (17 > 

ie{s-s^} 

As was the case for Eq.(14), we choose to simplify Eq.(16) 
in the following manner: 

F(Sf) = P(G!)[(1 - pP)f(S - Sp c ) 

-A 2 (l — pP) log (1 — PP)\ 

+P(G 2 )[(l-pP)f(S-Sp )c ) 

—A 2 (1 — /?j 2) ) log (1 - pP)\ 

-y(/?f - PP)( log(l - PP) - log(l - pf)) ( 18 ) 

Thus, for any set S 3 , we must find the sufficient conditions 
for the existence of diminishing property for both functions 
described in Eq.(14) and Eq.(18). To do so, we will evaluate 
their sufficient conditions and then find their intersection 
as the final conditions (assuming they do not negate one 
another). 

Theorem III.l. The objective function defined in Eq.(9) is 
submodular if 

(1) g{Sj,S w )<0 

(2) q(S J ,S w ,S r ,S t )< 0 

(3) \q(Sj,S w , S r , St)| > max{Z 4 , Z 2 , Z 3 , Z 4 ), 

Zl = A 2 log Uj' + y log (w'x'), 

Z 2 = A 2 log x' + -pjps lo g 
P\Gi) 


Z 3 — A 2 log LO 2 + 

Z 4 = A 2 log X 2 + 
(J = 1 + 


Ai 


P(G 1 ) 

Ai 


log (W 2 X 2 ), 
log (UJ 2 X 2 ), 


P(G 2 ) 

^11 

TtMi (ttli + TTMi ) 


x' = i + 


7Ti 2 


TtM 2 (tTi 2 + 7Tm 2 ) ’ 
TTll 


^ = 1 + rV 

1 7T]_ 2 


(19) 


for all possible sets S w C Sj anc/ .S) C ,S' r ant/ SV C ,Sj. 


The proof for this theorem is presented in the appendix 
under Theorem III.l. The proof simply consists of writ¬ 
ing down the diminishing returns property inequality and 
finding sufficient conditions so that each positive value’s 
coefficient is also positive. 

IV. Submodular Solution 

In this section, we offer an algorithm specifying how 
the problem formulated in Eq.(9) could be solved in 
a polynomial manner. It is important to note that this 
algorithm only offers a solution for when N = 2. However, 
even in the case of a binary output, an exhaustive search 
method would result in a 2 M complex problem. We thus, 
first offer the algorithm and then expand upon how the 
complexity of the solution has been compromised. Finally, 
we offer an example to showcase the relationship between 
solution error and complexity. 

A. Solution to Eq.(9) 

To offer our algorithm we define the objective argument 
in Eq.(9) as T{SP) and the constraint inequalities in 
Eq.(10) and Eq.(ll) as U(Sp) and W{SP). 

We then can write: 

Algorithm 1: Submodular Function Solution to the problem 

as described in Eq.(9) 

1. Let Si = argmax e&x= r! m -,T[Si = {e}] while 
U(Si = {e}) and W(Si = {e}) both satisfy the 
constraints. 

2. If there is an element e £ X \ Si such that 
T[Si + {e}] > T[5r] and H(5i+{e}) and W(Si + 
{e}) both satisfy the constraints, let Si = Si +{ei}. 

3. If there is an element e £ Si such that T[Si \ {e}] > 
T[Si] and {/(Si — {e}) and W(Si — {e}) both satisfy 
the constraints, let Si = Si — {e}. Go to Step 2. 

4. Return maximum of T[Si] and T[X \ Si]. 


Here, we know that at the very last step T[Si] = T[X\Si], 
Now we opt to calculate the complexities of this method. 
Steps 2 and 3 could repeat (.M — 1) + (M — 2) + ... + 1 = 
m{ai+i) t j mes eac j 1 w j 1 q e ever y item could be removed 
and thus replaced a total of 2 M times. Thus the total com¬ 
plexity of steps 2 and 3 is equal to M 2 (M +1) = 0(M 3 ). 
The complexity of step 1 is also equal to M. Thus the total 
complexity of the solution is equal to 0(M 3 ). 

This polynomial solution simply makes certain the max¬ 
imal and minimal functions obtained are at least 0.432 
and at most 2.315 times the optimal objective functions 
respectively. This range of error occurs because in this 
method, we are removing and adding members from and 
to the set Si one by one. Thus, at each decision point 
we are making one locally optimal decision. However, 
it is widely known that a locally greedy method is not 
necessarily globally optimal [9]. 

V. Numerical Examples 

We aim to offer the reader two problems where we are 
hoping to use the results gathered in Theorem III.l to 
first find a proper utility function given the specifics of 
each case. We will then follow Algorithm 2 and compare 




its results with that of an exhaustive search to compare 
the two methods in terms of complexity and exactness 
of the solution in both problems. In both examples, we 
assume M = 6 ,N = 2,Ai = 10, A 2 = 0.1, Ith = 
0.8, C = 3000 and that f(S) = /(|Sj) where |5| represents 
the cardinality of a set S. Furthermore we assume that 
P{G\) = 0.4, P(G 2 ) = 0.6. The only difference between 
the two examples would then lay in their respective prior 
item distributions. 

Note: In our following derivation of a desirable cost 
function we focus on developing a quadratic cost function. 
Such an assumption might appear to be extremely limiting 
to our class of functions at first sight. However, as is 
further explained in [10], such an assumption is quite 
understandable and rather desirable seeing as how in many 
economic models, functions are written as extensions of 
quadratic function and thus later calculations are simplified 
while still maintaining the essence of a cost function. 

A. Cost function for Table I 


Table I 

Example 1 for Item Probability distribution 


Hypotheses 

ei 

e2 

e3 

e 4 

es 

e6 

Hi 

0.11 

0.11 

0.09 

0.10 

0.10 

0.49 

h 2 

0.01 

0.01 

0.51 

0.02 

0.03 

0.42 


One of the simplest utility functions which could satisfy 
the conditions presented in Theorem III. 1 is a quadratic 
function in the form of f(S) = (IIS'! 2 + b\S\ + C. By 
calculating the 1 st and 2 nd order differences we can see 
that they follow the form of t/(|Sj) = 2a\S\ + b — a 
and q(S) = 2 a respectively. We could then develop the 
sufficient conditions for the utility function f(a) to satisfy 
Theorem III. 1 as developed for Table I. Given the specific 
values of lambda i and A 2 , one possible utility function 
will be in the form of /(|<Sj) = — ll|Sj 2 + 20|Sj + 41. 

B. Cost function for Table II 


Table II 

Example 2 for Item Probability distribution 


Hypotheses 

ei 

62 

63 

64 

es 

e6 

Hi 

0.10 

0.13 

0.23 

0.28 

0.17 

0.19 

h 2 

0.19 

0.12 

0.20 

0.20 

0.08 

0.21 


Following the same method, we could easily show that 
the same utility function /(|S|) = —Ills'! 2 —b20|S| +41 
satisfies the sufficient conditions gathered in Theorem III. 1. 

C. Solution Comparison 

The results of running Algorithm 1 on the two scenarios 

have been gathered in Table III. Each cell represents the 

maximum overall utility achieved in either case by each 

method where once again it is obvious that the probability 
distribution plays a major role on the exactness of the 

solution compared to the utility function -which is the same 
in both cases. Also of interest is the negligible loss of utility 

at 48 ' 48 ~ 7 o 8 27 ° 27 = 0-0012 while a desirable cost reduction 
from NP complex to polynomial has taken place. 


Table III 

Solution Exactness Comparison for Problem Formulated in 
EQ.(9) 


scenario exhaustive search solution submodular solution 

1 47+8 47+8 

2 48.7027 48J6 
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VI. Appendices 


A. Theorem 111.1 

Proof By first imposing the diminishing returns property 
to F(Sj) as formulated in Eq.(14) we see it necessary to 
have: 


«* 1 {-J 5 (G 1 )9(4 1) +W C. * * * * * (1) ,Si 1) + W (1) ) 

+Ai(log (a_e 2 + cr x f) — log («a 2 + a x 2 )) 
p(P(G i)A 2 + Ai)(log (a Bl + <+ei) — log (ola, + <+£,))} 
+<*xA~P(G 2 )g{Sf + {a:} ( “\ sf + {x}^) 

+Ai (log (a Bl + oi Xl ) - log (a a, +«n)) 

+ (P(G 2 )A 2 + Ai)(log (oib 2 + Q. x f) — log (<31,42 + <+£ 2 ))} 



+a BAl {-P{G 1 )g{S% ) + W (1) ,4 1} ) 

+Ai(log («b 2 +a X2 ) - log a B2 ) 

+(P(Gi)A 2 + Ai)(log(a Bl + a Xl ) - loga Bl )} 
+a BA2 {-P(G 2 )g{S {2) + {x}^,S {2) ) 


+Ai (log (a Bl + a Xl ) - log a Bl ) 

+(P(G 2 )A 2 + Ai)(log(aB 2 + a X2 ) - log a B2 )} 

+a Al {-P{G 1 )q(S ( B ) + {x}M,S% ) ,S ( 2 ) +{4 (1) ,5i 1} ) 


+Ai log (- 


a A o 


a B2 


a A2 


L ) 


+ (P(Gi)A 2 + Ai) log (- 


C 2 &B2 
&Ai &Bi + &xi 
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a. A! + ol x i Q!5 1 

+a^ 2 {-P(G 2 ) g (4 2) + {*}( 2 \4 2) ,4 2) + {4 (2) ,5i 2) ) 


+Ai log (- 


a Al 


a Bl 


a A i 

+(P(G 2 )A 2 + Ai) log (- 
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a A2 


OL B 1 

ftfl 2 + Q I2 


«a 2 


ob 2 


)}>0 

( 20 ) 


Now we attempt to find the sufficient conditions for 
Inequality (20) to hold true. 

If we assume g(S v S w ) < 0. S w C S :j we will satisfy 
the positivity of a Xl , a X2 , a BAl , and a BA _ 2 coefficients. 
It could further be seen that if we define 


u 1 = mini 


OLA i ol b i + a x 


OL A 1 


ttfli 


( 21 ) 


and 


X 1 = min{ 


ola 2 


ola 2 




CtB 2 + tta; 
OL B2 


( 22 ) 


then as long as q(Sj, S w , S r , S t ) < 0, S'™ C 5,-, S t C 
S r ,S r C 5j and 


M> 

maa;{A 2 log w + * log (wx), 

P(Gi) 

X ' 2 l0g X + P{Go) log l 23 ) 

we will have satisfied the diminishing returns property 
inequality. In the next step we attempt to calculate the 
values of uj and We further note that u> and \ represent 
the same set functions under different hypotheses, we could 
thus find the maximum value of either and then simply 
change the hypotheses index to derive the other. In the rest 
of this proof, we aim to find the maximal value of u. 

Since we are aiming to detect a lower bound over 3 
interconnected variables a Xl , a BAl and a Al , we need to 
show that the Hessian of the function ui is always negative 
meaning the value we find for w is a maximum. However, it 
could be seen that such a relationship does not necessarily 
hold true. We thus choose to find an upper bound for such 
a lower bound rather than deal with it directly. We will 
have: 


J{ol A i, a Xl ) — 1 + 


< 1 + 


ol Ai {ol Ai + a BAl + ol Xi ) 

1 1 


_ OLxi^BAi _ 

ol Ai (ql Ai + a BA i + a Xl ) 

< i + 


0LA 1 {0LA 1 + OL Xl ) 


= 1 


ol a i ol a i + a x 


J (ql A i , OL Xl ) 


(24) 


We now aim to maximize J'(a Al ,a Xl ). To do so, we 
first need to show that the Hessian of J ' is always negative. 
We will have: 




d 2 P 
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[' — j! 

^19. — «^9 


da A i 2 °l A i 2 ( a Al +a Xl f 
d 2 J' 2 


J ' 22 ~ 9a,, 2 


da Al da Xl (a Al + a Xl ) 3 
d 2 J' _ _ 2 

(o Al + a Xl ) 3 


(25) 


It could then be seen that J' n J 22 — J[ 2 J 2 \ — 0 meaning the 
function J' has a global maximum that could be found. To 
find this maximum we use the Lagrange Multiplier method 
where we hope to maximize: 


ol Ai a Al + ol Xi 

\{a Al + a Xl - K) (26) 


It turns out that J' is maximized when we have a Xl = ~ \, 
and a A1 = 7TMi and the resulting J' is: 


'^ , ('B'Mi: 7r li) — 1 + 


7TMi(7T h + TTMn) 


(27) 


Finally by reiterating u> using the new values we could 
see that 


u> < 1 + 


nii 

TTMi (tTIi + TTMi) 


It follows that 
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X < 1 + 


7Ti 2 

7r M 2 ( 7r l 2 + 7Tm 2 ) 


(29) 


Thus, for the function F(Sj) as described in Eq.(14), we 
have the following sufficient conditions: 


(1) g(Sj,S w )< 0 

(2) q(Sj,S w ,S r ,S t ) < 0 

(3) \q(Sj,S w ,S r ,St)\ > 

max{\ 2 log J + Xx log (wY)> 

P(Gi) 

A 2 log x! + r ul, T log YxO} 

F(G 2 ) 
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Now we aim to carry out the same process for F(Sj) as 
formulated in Eq.(18). By imposing the diminishing returns 
property, we see it necessary to have: 
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maximum. However, it could once more be seen that such a 
relationship does not necessarily hold true. We thus choose 
to find an upper bound for such a lower bound rather than 
deal with it directly again. We will have: 


Jli&An ot x i) — 1 + 


ot Xl a Bj 4i 

(1 - a Al )(l - a Al ~ a BAl ~ a Xl ) 


< 1 + 


= 1 + 
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Following the same logical steps, we could see that J 2 is 
maximized when we have a Xl = ni 1 and a Bl = 0 and 
the resulting J 2 is: 


J 2 (Oi tr 1;L ) — 1 “t~ z (35) 

1 — 7T 1:l 


Finally by reiterating w 2 using the new values we could 
see that 


Now we attempt to find the sufficient conditions for 
Inequality (30) to hold true. 

Once again, if we assume g{Sj,S w ) < 0 ,S W C Sj we 
will satisfy the positivity of a Xl , a X2 , a BAl , and a BA2 
coefficients. 

It could further be seen that if we define 
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It follows that 


and 


X2 1 


min( 


1 


1 ~ «a 2 

O^A 2 ^X 2 


1 — a B2 — a X2 
1 — a B2 


(32) 
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(37) 


then as long as q(Sj, S w , S r , St) < 0,S W C Sj,St C Thus, for the function F(Sj ) as described in Eq.(18), 
S r ,S r C Sj and we have the following sufficient conditions: 


M> 

max{\ 2 log to 2 + ' * log (uj 2 \ 2 ), 

P(Gi) 

A 2 log x 2 + log (w 2 X 2 )} (33) 

F\G 2 ) 

we will have satisfied the diminishing returns property 
inequality. In the next step we attempt to calculate the 
values of w 2 and y 2 . We further note that w 2 and X 2 
represent the same set functions under different hypotheses, 
we could thus find the maximum value of either and then 
simply change the hypotheses index to derive the other. In 
the rest of this proof, we aim to find the maximal value of 
c o 2 . 

Once again, since we are aiming to detect a lower 
bound over 3 interconnected variables a Xl , a BAl and a Al , 
we need to show that the Hessian of the function u 2 is 
always negative meaning the value we find for w 2 is a 
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Now that we have found the set of sufficient conditions for 
F(Sj) in both cases of Eq.(14) and Eq.(18), we can find 



the overall set of F(Sj) in all cases as: 

(1) g(Sj,S w )< 0 
(2) g(^-,5 w ,5 r ,5 t )< 0 
(3) | q(Sj,S w , S r , S t )| > max(Z 1 , Z 2 ,Z 3 , Z 4 ), 

Zi = A 2 logo;' + - log (w'x'), 

Z 2 = A 2 log x' + ^ r , . log (w'x'), 
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